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Policy Analysis for California Education 


Executive Summary 


California's school system is under tremendous long-run fiscal pressure; allocating resources 
efficiently is therefore paramount. Efficient allocation means more money spent on the most 
effective policies and interventions; less waste; and ultimately better outcomes for students. 
Economic analysis—making sure districts and schools are spending their budgets wisely—is the 
method used to identify effectiveness and efficiency. This method responds to the question 
educational professionals face: Am / making the most efficient decisions given the resources | have 
and the goals | need to meet for my students? 


In this report, we look at the role of economic analysis in decision-making in California's 
school system. Our goal is to show how economic evaluations can be more straightforward, 
more useful, and more influential for decision-making. Economic analysis is not often performed 
within educational institutions. We attribute this to misconceptions about economics as 
“cost-cutting” or narrowly focused on money rather than on how well students learn. But there 
are some legitimate and important barriers to performing economic analysis. Too often, it is 
not sufficiently useful for or relevant to the decisions that education professionals must make. 


In response, we describe cost-effectiveness analysis (CEA), one of the primary methods 
for economic evaluation. We provide a summary of this method, along with templates for 
framing the research analysis. We include a worked-through example to illustrate the key steps in 
performing CEA. We also describe case studies where such analyses have been performed that 
show the broad applicability of the method. 


We emphasize that CEA is: (a) about ensuring resources are used to meet districts’ and 
schools’ own objectives; (b) a joint effort with education professionals; (c) focused on whatever 
helps students learn and develop; and (d) responsive to institutional, cultural, and political 
preferences. 


We respond to the misconceptions and barriers that hinder economic research in 
California. To be more influential, economic analysts should: (a) make clear that, even as there is 
a cost to performing economic evaluation, there is also a cost to not performing it; (b) focus on 
policies and programs that require significant funding; (c) formally investigate how any results are 
relevant to the conditions faced by decision makers; and (d) explicitly consider how resources 
might actually be allocated towards a program that has been identified as cost effective. 


If analysts follow these steps, decision makers can feel more confident that economic 


analysis is justifiable and worthwhile. The end result should be better spending, greater 
accountability for decisions, less waste, and more learning by California students. 
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Economic Pressures on California's School System 


California's schools and districts face significant and cumulative financial pressures from 
many directions (Perry et al., 2020). Cost pressures are increasing in the present (because of 
student needs, e.g., for special education) and in the near future (from pension commitments). 
Revenue pressures are also significant, with falling enrollments and with erosions in the tax base 
from lower incomes and restrictive tax laws. Even before these pressures began to increase, 
resources for public schools in California were chronically inadequate.t In addition, the COVID-19 
pandemic has added an extra layer of pressure: costs have increased (e.g., for sanitation), even 
as in the short term significant extra funds have been allocated as part of the American Rescue 
Plan. Emphatically, education professionals across California face tremendous pressures to make 
decisions that are—given the resources available—efficient and cost effective. 


The role of efficiency criteria in making decisions has been heightened by recent policy 
changes to the Local Control Funding Formula (LCFF). Since 2013, each California school district 
has had to produce an annual Local Control and Accountability Plan (LCAP). This plan should 
describe the district's educational objectives and—critically—how its expenditures are matched to 
those objectives. Statements about educational progress are now reported jointly with budgetary 
information; the intention is to make the link between resources and outcomes more explicit. 

At the state level, the California School Dashboard displays quantitative evidence on how well 
schools and districts are performing. 


Education professionals have responded positively to these efficiency-promoting policy 
changes. School principals have taken the opportunity of new LCFF discretionary or supplemental 
grants to align their goals more closely with resource use (Koppich, 2019). As well, chief business 
officers have undertaken more direct roles in education decisions, using their LCAPs to change 
spending decisions and to link these decisions more directly to educational outcomes (Willis et al., 
2018). County officers of education have also become more involved in resource allocation.? 
However, still more direct links between resources and outcomes are needed so that LCAPs can be 
fully informative. 


1 For a full research compendium and evidence base, see Getting Down to Facts (https://gettingdowntofacts.com); on funding 
inadequacy, see Imazeki et al. (2018); on enrollment patterns and achievement, see Brighouse et al. (2018). 

? For some groups, however, the development and application of efficient decision-making has been more challenging. On the 
role of the California Department of Education and the introduction of the California Collaborative for Educational Excellence 
(CCEE), see Perry et al. (2020). On the challenges facing Sacramento City Unified School District, see Hahnel and Melnicoe (2019). 
Stakeholder engagement and school board contributions have been modest; see Willis et al. (2018). 


Cost-Effective Decision-Making for California's School System 


Policy Analysis for California Education 


Fundamentally, efficiency criteria need to be applied more widely in decision-making 
because public funds for education are highly constrained. Resources must be allocated across 
competing priorities.* 


Necessarily, a method should be used to make decisions. Which programs should be 
funded? If budgets are reduced, which programs should be eliminated? Or, if funding goes up, 
where should the extra money be spent? Too often, decisions are made “historically’—that is, 
doing things the way they have always been done, without any deliberate, methodical inquiry into 
whether this way Is efficient. 


In fact, formal methods for evaluating the efficiency of educational policies are well 
established (Levin et al., 2018). One approach is benefit-cost analysis, a policy evaluation method 
where all impacts and consequences are measured in dollars. The other approach Is cost- 
effectiveness analysis (CEA), a policy evaluation method where the resources to effect change are 
compared to the size of that change. For education decision makers, CEA is more appropriate 
than benefit-cost analysis. CEA responds to the question: Am | making the most efficient 
decisions given the resources | have and the goals | need to meet? 


In this report, we respond to the need for efficient decision-making within the context 
of economic pressures and statewide policy reform in California. Our goal is to promote the 
capacity of education professionals throughout California to evaluate the efficiency of policies 
and programs by showing how CEA can be applied. We begin with a review of the context 
for decision-making across the California school system and highlight the many obstacles to 
making efficient decisions. Next, we describe the method of CEA, offering a how-to guide for 
school district personnel and other educational professionals. Then, we apply the method to a 
set of specimen case studies; these illustrate how economic analysis is performed in different 
educational settings. Finally, we look at the challenges in making this research relevant for 
decision makers. 


This report is intended for all professionals across California's education system who make 
resource allocation decisions. Primarily, this group is composed of school district officials such 
as superintendents, chief business officers, and heads of departments and strategic planning 
offices. The group may also include professionals within the offices of the California Department 
of Education, representatives to the State Board of Education, and school principals. For these 
professionals, this report is motivated to make economic evaluations more straightforward, more 
useful, and ultimately more influential for decision-making. 


> These constraints held before the COVID-19 pandemic but they have become imperative as the pandemic has persisted. With 
extensive distance learning, districts must decide, to take a few examples: how much to spend on online programs; what class sizes 
are optimal; how teachers should be trained; and what extracurricular services are affordable. The pandemic is also likely to 
increase the need for education that supports students’ social-emotional development. 
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Economic Analysis in K-12 Education in California 


Education professionals across the state’s school system can apply economic research and 
analysis to make more efficient decisions. But—based on our experience and on investigations 
spanning several decades—they mostly do not do so in any systematic fashion. This failure to apply 
economics is partly due to a number of misconceptions about what economic analysis is and Is 
partly the result of several significant barriers that decision makers face. 


Misconceptions about Economic Analysis 


There are a number of important misconceptions about the role of economic analysis 
in decision-making. Too often, these misconceptions mean economic analysis is immediately 
rejected. 


Economic analysis is not about “cost-cutting,” identifying “savings,” or “cost 
containment.” It is not about spending less money on education; it is about spending the 
available money in the best way to achieve the goals the education system has set for itself. 
Indeed, economic analysis might reveal that more resources should be committed to a 
particular program. 


Economic analysis is not something accountants or finance officers do independently 
with budget statements. It is not about satisfying accounting conventions or meeting financial 
targets; it is about using resources in ways that make sense from an educational or pedagogical 
perspective. Education professionals—not accountants—are best placed to analyze what Is 
educationally or pedagogically effective and so have an important role in economic analysis. 


Economic analysis is not about accounting for educational materials such as paper 
supplies or computers. It is not about making sure that equipment purchases are made in 
accordance with general accounting principles; it is about all the resources used to promote 
learning. The priority for economic analysis is therefore on the most important resources—and, 
in particular, the time—of teachers instructing students in classrooms. 


Lastly, economic analysis is not about modelling districts and schools as if they were 
factories intended to maximize “production” or “profits.” Districts and schools are institutions with 
their own cultural and political environments in which education officials, teachers, local unions, 
and parents are all influential. These cultural and political conditions do influence decisions. But 
districts and schools must still provide resources for programs and make choices across programs; 
districts and schools do not want to deliberately waste resources on programs that do not help 
them achieve their objectives. By providing a rigorous evaluation of programs, economic analysis 
can contribute to cultural and political dialogues among groups. It can bring groups (political, 
cultural, or other) together to discuss the trade-offs involved in decisions. 


Cost-Effective Decision-Making for California's School System 


Policy Analysis for California Education 


Barriers to Economic Research 


There are some important barriers to performing economic analysis.4 Some are general 
barriers that hinder the application of any research activities; then there are barriers specific to 
ECONOMICS. 


One very basic barrier is the shortage of time and capacity to undertake economic 
research. Evaluations take time and need to be adequately funded; they require personnel trained 
in evaluation methods; and valid data needs to be collected and analyzed. Many districts may 
feel that they cannot afford—or do not have the capacity—to undertake economic analysis.° 


One fundamental barrier is that, too often, any research inquiry is perceived as not 
concretely helpful for decision makers. The information produced from research is often not in 
the form education professionals need It to be in. For decision-making, information needs to 
be relevant, conclusive, and timely. Districts and schools need to make so many decisions about 
so many different programs and policies and iterations thereof that it may not be practicable 
to conduct evaluations or find rigorous evidence for all of them. Evaluations, furthermore, may 
not yield clear answers on which programs are effective and efficient. Research findings—with 
multiple outcomes and no statement of perspective—are often difficult for audiences to interpret 
or “take away.” And even if economic analysis is performed, it may be too late: educational plans 
are already committed and cannot be changed.® 


Research may, moreover, be regarded as irrelevant if it is not sufficiently sensitive to the 
local contexts of districts and schools. For example, supplementary reading programs may 
only be effective for districts with high levels of disadvantage; teacher incentive schemes may 
only work if there are shortages at a local school. Context is especially important for economic 
analysis; along with educational contexts (such as class size or student disadvantage), there 
are economic constraints—amounts of funding per student, district funding formulae, and 
union contracts, as well as local price levels. Even if interventions are equally effective across 
all students, their costs need not be. For example, larger districts may be able to buy textbooks 
with bulk discounts; urban districts may have to offer more generous teacher incentives because 
living costs are higher. This barrier creates a distrust of research and a preference for professional 
experience in local contexts (or for continuing with “business as usual’). 


4 See, for example, Watling Neal et al. (2018). We focus on the context of education systems but recognize that most of these 
barriers are not specific to education professionals but apply across many professions. Also, we assume that the research has been 
performed to an acceptable methodological standard. 
° Gathering data is often perceived by schools and districts as both cumbersome and unnecessary; often they are focused more on 
their own bottom lines than on a more abstract economic definition or on a broader social perspective. As such, costing exercises 
are thought of as being too time intensive to be worthwhile. 

© Educational institutions are often very conservative, with a strong bias towards the status quo. New programs and policies must be 
demonstrably effective and impactful before they will be considered, even in circumstances where there is no evidence in favor of 


the status quo. 
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Institutional barriers may also exist. In some contexts, no single decision maker or 
professional has control over resources or redeployment of resources (or multiple stakeholders 
cannot reach agreement). For example, districts may contribute funding as a supplement to 
federal funding (or external grants). Resource decisions are often constrained by rules, such as 
union contracts or funding formulae or legal stipulations. A related challenge is that resources 
are typically “sunk” into existing programs. For example, even if California Math is the most cost- 
effective textbook for Grades 3-5, schools have already bought different textbooks; teachers 
cannot be instantly hired or replaced.’ Or, if a student has an Individual Education Plan (IEP), 
resources cannot be reallocated to alternative uses even if those uses are more cost effective. 
When there is no clear authority for decision-making or when resources cannot be reallocated, 
economic research is unlikely to help. 


The Need for Economic Analysis and Research 


These misconceptions and barriers are important: they restrict the application of economic 
analysis. But they do not mean that economic analysis is unimportant. 


Fundamentally, the dollar amount of spending on education requires economic evaluation. 
Annually $450 billion is soent on K-12 education across the state; it is imperative that this money 
be spent efficiently. Equally powerfully, this analysis is needed because of the economic conditions 
facing the California school system. 


The financial state of California's school system raises the stakes for economic analysis.® 
Spending per pupil in California is 20 percent lower than spending nationwide, adjusting for local 
prices. This is reflected in educational resources: teacher to student ratios are 40 percent larger 
than they are nationwide; counsellor to student ratios are 45 percent larger. Money for teacher 
salaries has essentially flatlined over the last decade. And looking across the state’s public funds, 
education is competing with—and losing out to—other public investments such as health and 
criminal justice. These constraints place real pressure on districts to make cuts in funding but to 
make these cuts in a cost-effective way.? 


The next two sections of this report are intended to enhance the use of economic analysis. 
Specifically, we describe cost-effectiveness analysis, the method for evaluating educational 
programs and policies against their intended objectives. Next, we provide case studies of CEA to 
show that this method can be practically and successfully applied within the California school 
system. The overall objective is to show how CEA is helpful and relevant for decision makers. 


’ For evidence on math textbooks, including California Math, see Koedel et al. (2017). 
8 See evidence from Hahnel (2020). 
° See Hahnel et al. (2020). 
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Cost-Effectiveness Analysis: Method 


CEA is an evaluation method that compares the educational effects of alternative 
educational policies, programs, or interventions in relation to the resources they require in order 
to be implemented.?° Box 1 summarizes the main idea of CEA. 


Cost-Effectiveness Analysis 


¢« Helps decision makers evaluate policies or programs when resources are scarce; 
* iS Comparative: it helps decision makers choose among alternatives; 
* produces a ratio of costs to quantified effects for each alternative; 


e across alternatives, lower ratios mean more cost effective. 


Essentially, CEA Is a comparative exercise—it ranks educational interventions. Thus, 
when undertaking CEA, education professionals should only compare interventions that: (a) are 
comparable—that is, they have the same objectives for the same student groups—and (b) are 
actual alternatives—that is, where only some interventions are to be implemented (and other 
interventions are therefore to be rejected). 


Valid information on costs and on effectiveness is needed to perform CEA calculations. 
Collecting this data and performing the analysis takes time and resources; these are unavoidable. 
However, having a formal protocol expedites CEA. Below, we outline the three steps of this 
protocol; we then present research templates as guides to gathering the required information. 


How to Measure Costs? 


Costs are all the resources used within a district or school to produce the desired set of 
educational outcomes. To accurately measure costs, the analyst should apply the ingredients 
method: each input should be deliberately itemized and then priced, regardless of its funding 
source. Often, the primary input is teaching staff; the cost is their time multiplied by the full 
compensation teachers receive for that time. Other inputs include the time of all educational 
professionals, materials, equipment, and facilities. Every ingredient needs to be tabulated 
and counted. 


10 This section draws heavily on Levin et al. (2018). Readers are encouraged to refer directly to this source for more detailed 


information. 
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Obtaining cost data is not simply a matter of looking at a budget statement. Budget 
statements are designed to satisfy accounting conventions; they do not separate resources 
used for specific interventions. Teacher pay, for example, is often included as a single budget 
line item that accounts for all instruction. However, teachers perform many tasks, including 
administrative and managerial work. Other ingredients, such as facilities, are not directly charged 
to an intervention. Also, resources are funded by multiple agencies, including state and federal 
governments. Finally, budget statements describe how resources should be deployed; costs are 
measures of how resources are actually used. Budgetary information may approximate costs, 
but it is important to verify this. 


Costs should be expressed per unit. For example, a new reading program may be 
reported as cost per student; a class size reduction policy may be reported as cost per school; 
and an online-learning platform may be reported as cost per district. With these units specified, 
effectiveness can be measured in equivalent terms (per student, per school, or per district). 


Education is a process; costs may be incurred at different times. For example, a new 
test for special education placement may mean extra resources for students during all grades 
of school; or, enhanced school readiness programs may mean fewer resources are needed for 
students struggling in first grade. These resource consequences are called “induced” or “indirect” 
costs. They are not strictly costs of the intervention (the new test or enhanced school readiness 
program) but they are still costs and—if they help to generate effects—they should be included 
in cost calculations. 


How to Define Effectiveness? 


Any metric can be used to measure educational effectiveness—its validity depends on 
the goals of the decision maker. Test score gains are often a priority, but there are many others: 
high school graduation; percentage of English learners (ELs) wno become proficient; school 
expulsion rates. For example, Figure 1 shows a set of academic domains on the Dashboard for 
San Francisco Unified School District.t But districts and schools may focus on nonacademic 
domains, including child health or social-emotional well-being. Any or all of these may be 
considered a valid measure of effectiveness. 


™ Even more expansive is the LCFF “State Priorities Snapshot Data File,"which lists over 60 educational metrics that might each be 
regarded as a valid measure of effectiveness. 
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a ey] 
Figure 1. Some Possible Measures of Effectiveness 


DISTRICT PERFORMANCE OVERVIEW 


San Francisco Unified 


Explore the performance of San Francisco Unified under California's 


Accountability System. 


Chronic Absenteeism 


Suspension Rate 


English Learner 


Graduation Rate 
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os, 
Orange Green | a | Green 
No Performance Color 
College/Career English Language Arts Mathematics Basics: Teachers, 
Instructional Materials, 

Facilities 
Green Green Yellow 
Implementation of Parent and Family Local Climate Survey Access to a Broad 
Academic Standards Engagement Course of Study 


District Details 


NAME 
San Francisco Unified 


Source. California School Dashboard (n.d.). 


ADDRESS 
555 Franklin Street 


San Francisco, CA 94102- 


5207 


WEBSITE 
http://www.sfusd.edu 


GRADES SERVED 
P-12 


The first task for the decision maker is to specify—in detail—what effectiveness measure 
will be used in the analysis. Economic analysis does impose some conditions. Essentially, any 
effectiveness metric must be expressed in quantifiable units. Preferably, one single metric should 
capture all the goals of the decision maker in relation to any policy. The metric should relate to 
all students, not just a targeted few, and it should be easy to understand. But, to emphasize: 
for the purposes of CEA, the decision maker should determine the effectiveness measure. 


Educational effects must be identified empirically so as to establish that the resources 
used did in fact cause the effects. A range of estimation methods may be employed, including: 
correlational or regression analysis; meta-analysis; experimental trials. Analysts may rely on 
evidence from external evaluators (such as the What Works Clearinghouse). Often, identifying and 
measuring effectiveness is a separate research task from calculating costs and then determining 


cost effectiveness. 


Evidence on effectiveness needs to be produced expeditiously—decision makers cannot 


wait years for this evidence. With timely evidence gathered on an ongoing basis in the longer 
term there is an opportunity for a cycle of continuous improvement: impact studies and cost 
analysis may be refined and developed over time to provide new policy-relevant guidance. 
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How to Determine Cost Effectiveness? 


To determine the efficiency of any intervention, its implementation costs (C) are divided by 
its effects (E). The result is a cost-effectiveness ratio (C/E; CE ratio), reported in dollars, for each 
intervention and for each comparison between interventions. 


The ratios yield a simple decision rule: cost-effective interventions are ones where the 
largest effects are generated for the lowest amount of resource. That is, the preferred intervention 
is the one for which the CE ratio is the smallest out of all the alternatives. As emphasized above, 
CEA is a Comparative exercise, where each alternative education policy is ranked based on its CE 
ratio. The strong presumption is that the most cost-effective intervention is preferred. 


Cost effectiveness Is not the only factor in educational decision-making. Affordability is 
essential: some interventions may be too costly even if they are the most cost effective. Equity is 
crucial: interventions may be made available to all students even if they are only cost effective for 
specific populations of students. Finally, perspective matters: interventions may be cost effective 
for a school but not for a district. Hence, CEA provides information that is intended to support the 
decision-making process rather than be followed as a rule regardless of context and local values. 


Cost-Effectiveness Analysis Templates 


Templates for effects, costs, and cost effectiveness may serve as a guide for analysts. 
magine a choice of three programs or interventions. Labelled here as X, Y, and Z, the programs 
might relate to: professional development (online, in school, hybrid) with the goal of improving 
teacher performance; or math curricula by different providers (SoringBoard Mathematics, Go 
Math!, CA Digits) with the goal of increasing math test scores in eighth grade; or indeed any 
intervention the district wishes to evaluate. Following these templates, the analyst should be able 
to determine which program is the most cost effective. 


For Template C (cost), the cost of each program should be calculated—separately for each 
program. Cost categories are personnel, facilities, materials, and other inputs; for each category 
the cost estimate should be assigned to the funder: district, school, or other, depending on 
how that resource is funded. Any external subsidies or user fees should be subtracted from the 
total ingredients cost to yield the net cost; this net cost must then be reported per unit (student, 
school, or district). The analyst should check: that all costs are identified; that costs are collected 
at the site level; how accurate budget data Is, if relied on for costs; and if all money transfers 
between agencies are properly accounted for. After these checks, the net cost per unit can be 
calculated accurately. 
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Template C. Cost Spreadsheet per Program 


Resource use per funder 


District KYel atofo) | Other 


Personnel: 
Managerial/governance, instructional, administrative 


Facilities: 
Rental cost, depreciation 


Materials and equipment: 
Instructional materials, IT, other equipment 


Other inputs: 
Transportation, food, and health services 


Required inputs from others: 
Matched external support, parental effort 


Total ingredients cost: 
Minus user fees, subsidies 


Net cost 


Units (number) 


Net cost per unit (C) 


Are all costs identified? 


Are costs collected at site level? 


Is budget data valid? 


Are transfers between agencies accounted for? 


For Template E (effectiveness), the independent effect of each program must be estimated. 
As a prompt, the analyst should determine the validity of the effectiveness metric used, asking if 
the metric captures the goals of the program and applies equally to all programs and all students, 
as well as determining that any effects are identified using a rigorous research method. If so, the 
effects (e.g., on teacher performance or eighth-grade math scores) can be written into Template E. 
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Template E. Effectiveness Metrics 


Program X Program Y Program Z 


Effect per unit le le E 


x Y Z 


How well does metric: 


Capture goals of program? 


Match across programs? 


Apply to all students? 


Is effect identified by 


Valid estimation method? 


For Template CE (cost effectiveness), the results on effects and costs from Templates 
C and E are combined. The CE ratio is then simply the costs for each program divided by the 
effects of that program. A straightforward decision rule is: select the program with the lowest 
CE ratio. Measured in dollars, the ratio can be thought of as the “price” paid for a given effect; 
lower prices are usually better. The analyst may want to directly compare two programs. An 
incremental CE ratio is calculated as the difference in costs divided by the difference in effects. 
This incremental cost-effectiveness ratio shows the “price” of moving from one program to 
another. As a prompt, the analyst should check if the CE ratio—or the rank implied by the CE 
ratio—is robust to alternative assumptions or data. Importantly, the analyst should ensure that the 
results are easy to explain and are informative for decision-making. 


Template CE. Cost-Effectiveness Analysis Results 


Program X Program Y Program Z 
Net cost per unit Cy Cy Cy 
Effect per unit Ee Ey ES 
Cost-Effectiveness ratio Cie, Cie. es 
Incremental cost-effectiveness ratio — (GAC WE se) (Ce) (==) 


Is the cost-effectiveness ratio: 


Robust to alternative data? 


Easy to explain? 


Useful for decision makers? 
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Step-by-Step Cost-Effectiveness Analysis for 
Local Control Accountability Plans 


The CEA method can be illustrated as a series of steps. These steps are itemized here, 
drawing on the terms used in LCAP reports (italicized in the headings below). Examples for 
each step appear below each heading in quotation marks. We emphasize that the cost and 
effectiveness measures must be derived using valid research methods; the numbers here are 
illustrative of the method. 


Potentially, these steps should be applied to each action included in each LCAP statewide. 
If performed more extensively, CEA would significantly increase the informative content of 
LCAPs and lead to continuous improvement in resource allocation within districts. 


Identify the Goal 


“Ensure all parents and community members are welcomed and engaged in the learning 
process.” 


Specify Measurable Outcomes in Relation to the Goal 
“Percentage of parents that strongly agreed or agreed that they feel welcome.” 
Describe Alternative Actions to Satisfy These Goals 


e« Parent Resource Centers — Family and Community Engagement staff and resources 

to support Parent Resource Centers throughout the District and build the capacity of 

parents and families to support student learning 

¢« Parent University — District-level staff and resources to facilitate the education of 

parents 

¢« English language instruction to parents — Community-Based English Tutoring program 

of adult English language instruction to parents 

« Parent Advisory Committee — improved communication with parents via the District 
website and social media, parent-teacher collaboration, phone message system, and 
staff for Communications and translation services, as well as other services 


” Examples are adapted from Bakersfield School District (2019, pp. 56-63). 
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Calculate Expenditures per Action 


These expenditures are typically reported as annual amounts in LCAPs. See Table 1. 


Table 1. Expenditures per Action 


Not dey a] Cost 


Parent Resource Centers $1,465,000 
Parent University $551,000 
Parent English Language Instruction $191,000 
Parent Advisory Committee $2,920,000 


For example, the parent university cost might be calculated as full-time faculty (including 
indirect costs) plus costs of overhead and facilities. For example, the English language instruction 
to parents is hours of tutoring plus instructional materials and facilities. These are itemized below 


in Table 2. 


Table 2. Ingredients per Action 


Action Parent tale l tam m-lale Loreto [= 
University Instruction 
Certificated Personnel Salaries $17,400 $37,800 
Classified Personnel Salaries $216,100 $83,100 
Employee Benefits $138,100 $44,900 
Books and Supplies $118,700 $20,400 
Services and Other Operating Expenditures $42,600 $4,800 
Other Title | Outgoings $17,100 _ 
Total Cost $551,000 $191,000 
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Describe the Overall Effectiveness of the Actions/Services to Achieve the Articulated Goal as 
Measured by the Local Education Agency 


For CEA, effectiveness should be quantified (not described, as it typically is in LCAPs). 
The effectiveness measure should be derived from a research study. For example: 


¢« Access to a parent university might be randomly assigned; the “effect” of the university 
could be quantified as the difference in feeling welcome (from a postuniversity survey) 
between parents assigned to the university and those in the control group. 

e« English language instruction to parents might be made available for third-grade 
parents; these parents’ levels of welcome could be compared to the levels of fourth- 
grade parents. 


Any causal method may be used to quantify effectiveness. Illustrative numbers are given 
below in Table 3 for the four selected actions so that we can calculate cost effectiveness. 


Table 3. Effects per Action 


Action Parents Feeling 
Welcome (gain) 
Parent Resource Centers 300 
Parent University 100 
Parent English Language Instruction 50 
Parent Advisory Committee 800 


Calculate Cost Effectiveness of Each Action 


This step is not included in the LCAP reports. However, it is the essential step in 
performing CEA. Ratios in Table 4 are calculated for purposes of illustration. 


Table 4. Cost-Effectiveness Ratios per Action 


Yeh d (oa) Cost Parents Feeling CE Ratio 
Welcome (gain) 

Parent Resource Centers $1,465,000 300 $4,880 

Parent University $551,000 100 $5,510 

Parent English Language Instruction $191,000 50 $3,820 

Parent Advisory Committee $2,920,000 800 $3,650 
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Decide Which Actions to Fund in Future Years 


Based on the illustrative results for step 6, parent advisory committees are most cost 
effective at making parents feel welcome; English language instruction is next most cost 
effective; parent university programs are the least cost effective. This information might be 
ncluded in LCAPs or considered when LCAPs are being compiled. 


With these CE ratios as an important factor, decision makers should nevertheless still take 
political, community, and professional views into account. 


Cost-Effectiveness Analysis Case Studies 


Case studies of cost effectiveness are helpful to illustrate how CEA is performed and to 
highlight the general problems analysts and decision makers may face when undertaking CEA. 
We present five case studies. Some are based on actual CEAs; others are “specimens” contrived 
to illustrate the possible circumstances faced by analysts. 


Changing the Math Curriculum 


For this case study, we use existing evidence on the effectiveness of math curricula to 
explore if it would be cost effective for schools to adopt a new textbook. 


Koedel et al. (2017) evaluated the effectiveness of four math textbooks popular in 
California. California Math (published by Houghton Mifflin) was identified as producing 0.05-0.08 
standard deviations more math achievement in Grades 3-5 than the other three textbooks.¥ 
At issue is whether switching to California Math would be cost effective for districts. Koedel et al. 
(2017, p.1) and other researchers claim that this gain can be achieved at “effectively zero marginal 
cost” but there is no extant evidence. For a CEA, we need to know the cost of moving from a 
generic textbook to California Math—costs must be calculated, not trivialized. For illustration, we 
consider the likely costs if a district switches from a generic math textbook to California Math. 


The resources needed to switch textbooks may not be trivial. Clearly, the district will have 
to purchase new textbooks, lesson plans, and assessment materials (and rescind or dispose of all 
prior materials). Reasonably, market prices can be used to estimate the costs for these materials. 
However, this is not the primary cost. Critically, training on the content standards of California 
Math is needed for teachers and education professionals (e.g., district officials, school managerial 


13 We do not intend the studies to be advocacy for any particular program or reform. 
The other three were: enVision Math California; California Mathematics: Concepts, Skills, and Problem Solving; and California 
HSP Math. 
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personnel, evaluators, and assessors). Conservatively, we might estimate one day of training in 
the content standards for each professional, so the cost will be one day’s compensation. 
Newly trained in the content standards, teachers will have to devise new instructional materials 
(lesson plans, assignments, and assessments); this may be a substantial time commitment. 


The time cost of new instructional materials will depend on factors such as: 
e how much new preparation is required per class; 
e« the number of new classes (teaching load) per math instructor; and 


e the duration over which the new materials remain valid. 


These factors vary across districts and schools but are critical for determining the cost— 
and therefore the cost effectiveness—of switching to California Math. 


Textbooks vary in how they affect achievement. But choosing a textbook 
is not just about effectiveness. The cost of switching textbooks depends 
critically on the resources required for new lesson preparation and how 
math teachers are deployed across schools. 


Helping Struggling Readers 


For this case study, we look at the cost effectiveness of reading interventions when funding 
is split across agencies. 


Reading Partners is a supplemental pull-out reading program for struggling students 
in Grades K-5; it serves students in over 50 elementary schools across California. National 
evaluations have identified positive and statistically significant impacts on reading proficiency: 
the program is effective relative to standard reading supports.’® 


Cost data on Reading Partners is available from research by Jacob et al. (2016) using 
the ingredients method at 19 schools nationwide. For standard reading supports, schools 
typically spend $1,700 per student; for Reading Partners, the cost per student is double, at $3,610. 
However, responsibility for providing the resources for Reading Partners is split between two 
groups: volunteers supported by AmeriCorps and school-site coordinators, supervised by program 
managers. Table 5 shows the ingredients used to implement Reading Partners. Notably, schools 
only fund $710 for Reading Partners; most of the funding comes from external sources, displacing 


8 Anecdotally, for each classroom hour of new math instruction, teachers report spending anywhere from 0.5-3 hours on 
preparation of instructional materials. 
© For information on the program, see readingpartners.org. For evidence of effectiveness, see Jacob et al. (2014). 
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the standard supports. This amount is less than schools spend on “business as usual.” From the 
perspective of the school and district, Reading Partners is more effective and lower cost—and 
therefore strongly cost effective—relative to standard reading supports. 


Table 5. Reading Partners: Resources and School Costs 


Cost per Student 


Full Resources School-funded 
Personnel: Reading Partners $690 $320 
Personnel: School staff $90 $90 
Facilities $300 $300 
Personnel: AmeriCorps $930 — 
Volunteers and transport $1,520 _ 
Materials/equipment S80 = 
Total Ingredients Cost $3,610 $710 


Source. Jacob et al. (2016); 2014 dollars. 


Programs that are demonstrably effective and (because of subsidies) have 
low costs to the school are highly cost effective. 


Increasing High School Completion 


For this case study, we look at the cost effectiveness of high school completion programs 
when costs and effects vary significantly across program sites. 


Talent Search is a federal program for middle and high school students from low-income 
families with potential to attend college but who may be at risk of high school dropout. Annually, 
federal spending on Talent Search is over $30 million across California, with over 75 sites across 
the state. There is evidence that Talent Search Is effective at increasing the rate of high school 
completion (Constantine et al., 2006). 


However, Talent Search is not a rigidly prescribed intervention. Colleges and schools offer 
a range of services to high schoolers, including academic/career counseling, field trips, financial 
awareness training, college tours, help with college applications, and college exam preparation. 
Students can participate in Talent Search for between 1 and 6 years in middle/high school. Thus, 
the resources allocated to each student can vary significantly. Districts and schools need to 
choose a version of Talent Search that is most cost effective. This imperative holds even though 
the program is federally funded. 
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Results on costs and effects of Talent Search are shown in Table 6. On average, across 
nine Talent Search sites, the cost per student was $3,630. The effect was an 8.8 percentage point 
increase in high school completion: if delivered to 100 students, there would be 8.8 more high 
school graduates. The resulting CE ratio is $41,250 per high school graduate (i.e., the program 
spends this amount to yield each new graduate). However, of interest for decision-making is the 
variation in cost effectiveness across the sites. The lowest cost site has a CE ratio of $24,160; 
the highest effect site has a CE ratio of only $10,480. These sites are much more cost effective 
than the average site. By contrast, high-cost sites and low-effect sites are not cost effective. 


Table 6. Talent Search: Cost Effectiveness Across Sites 


Average across 9 sites $3,630 0.088 $41,250 
Site: 
Lowest cost $2,730 0.113 $24,160 
Highest cost $5,190 0.09 $57,670 
Most effective $2,960 0.273 $10,480 
Least effective $2,800 -0.035 -$80,000 


Source. Bowden and Belfield (2015); 2010 dollars. 


Even if districts do not provide funds for a program, it is important to 
choose the most cost-effective version of that program—and that choice 
is not necessarily the lowest cost or highest effect version. 


Helping Children Stay Healthy 
This case study shows how CEA has a broad application within school environments. 


Food, Health, & Choices (FHC) is a 24-lesson nutrition education curriculum for elementary 
school children. Composed of 24 lessons over one school year, the curriculum emphasizes dietary 
changes (reducing sugars and processed food, increasing fruits and vegetables) and behavioral 
changes (more exercise, less screen time). The objective is to reduce the childhood obesity rate. 
With a randomized controlled trial, Graziose et al. (2017) evaluated FHC across 77,678 fifth-grade 
students in 2,953 classrooms in New York City public schools. Treatment students received FHC; 
control group students received “business as usual’—which, in New York City at that time, was no 
formal nutritional education. 


Graziose et al. (2017) performed a CEA of FHC. The results are summarized in Table 7. 
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Table 7. Cost-Effectiveness Analysis of Food, Health, and Choices 


Participating students 77,678 
Obesity cases averted (E) 639 
Costs: 
Teacher preparation time $3,685,300 
Trainers (16 FTEs) $1,520,000 
Teacher professional development $1,229,000 
Curriculum materials $1,034,600 
Coordinators (4 FTEs) $380,000 
Other teachers $500,000 
Other personnel/inputs $190,000 
Total Costs (C) $8,537,900 
CE Ratio (C/E) $13,360 


Note. FTE = Full-Time Equivalent. 
Source. Graziose et al. (2017); 2010 dollars. 


As shown in the top panel of Table 7, FHC is effective at reducing childhood obesity. 
By age 10, the FHC participant group had obesity rates that were 2 to 4 percentage points lower 
than the control group. For CEA, yields are often easier to explain than rates, so that effect is 
translated into 639 obesity cases averted across the student population. The middle panel shows 
the cost of FHC: significant resources were needed for teacher preparation time and professional 
development, as well as for trainers and curriculum materials. In total, FHC cost $8.54 million 
across all students. Therefore, the CE ratio for FHC over control is $13,360. 


For decision makers, this ratio can be interpreted thus: the district (or funder) is paying 
$13,360 for each averted case of obesity. This dollar figure seems low but additional information 
may be needed to declare FHC cost effective. First, are other obesity prevention programs 
available with lower CE ratios? Second, does obesity adversely affect other outcomes, such as 
academic achievement? Third, does obesity prevention save districts or schools money 
(e.g., on medical costs) in subsequent grades? Fourth, does the program benefit students who 
were not at risk? Issues like these are relevant for efficient decision-making even when a 
CE ratio appears low. 


School districts may fund health and wellness programs; cost-effectiveness 
analysis is valid for investigating if such programs are efficient but other 
information may be needed to make a final determination. 
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Hiring Teachers Optimally 


A case study of teacher retention policies illustrates the importance of specifying 
effectiveness measures for cost effectiveness. 


California—as with many states—faces severe teacher shortages, with many “hard-to-staff" 
positions and low teacher retention (Darling-Hammond et al., 2018). Numerous policy responses 
have been proposed; these typically fall into two categories. Teacher remuneration may be 
increased, either through direct pay increases, retention bonuses, loan forgiveness plans, or 
indirectly via improved working conditions. Alternatively, teacher training may be enhanced, either 
through better programs, within-school mentoring, or residency programs.” 


These policy responses may have many varied consequences (Feng & Sass, 2018; Strunk & 
Zeehandelaar, 2015). Yet CEA requires the effectiveness measure to be a single metric. However, 
this requirement is sometimes helpful: it forces the evaluation to focus on a single, critical metric. 
This focus is illustrated in the evaluation of one short-lived policy to boost teacher retention in 
California. 


The California Governor's Teaching Fellowship (GTF) program was implemented from 
2000 to 2002; it was then suspended because of concerns over affordability and competing 
priorities. GTF, which has been evaluated by Steele et al. (2010), was designed to attract newly 
licensed teachers to low-performing schools and retain them for at least 4 years. The fellowship 
was selective, with 1,100 awards over 2 years. Each award was $5,000 per year of service, 
approximately a 15 percent boost over average starting salaries. 


An approximate CEA is reported in Table 8, based on the Steele et al. (2010) results. 
Although 1,100 teachers participated in GTF, many of those teachers would have accepted 
positions in low-performing schools without the award; many participants quit within a few years. 
Thus, for the economic evaluation, effectiveness was strictly defined as the increase in total 
years of teaching in low-performing schools within four years. Steele et al. (2010) calculated 716 
additional years of teaching from GIF. The full costs of GTF were not recorded; an estimate is 
shown in Table 8. The awards totaled $7 million and there were also administrative costs and costs 
for fellowship selection. These costs amount to approximately $7.98 million. 


” A third category is expansion of the teacher labor market to allow reentry by retirees or alternative credentialing. 
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Table 8. Cost-Effectiveness Analysis of the Governor's Teaching Fellowship 


Participating teachers 1,100 
Years teaching in low-performing schools (E) 716 
Costs: 
Fellowship awards $7,000,000 
Administration? $840,000 
Fellowship selection? $140,000 
Total Costs (C) $7,980,000 
CE Ratio (C/E) $11,150 
Source. Adapted from Steele et al. (2010); 2002 dollars. 
Note. * Estimated from government transfer programs. ° Estimated. 
School districts were therefore paying $11,150 (the CE ratio for GTF) to obtain one extra 


year of teaching in a low-performing school. As teacher salaries at the time were approximately 
$40,000, the GTF was equivalent to a 25 percent bonus to these teachers each year. The CE 
ratio is therefore indicative of the challenge of “hard-to-staff" teaching positions in California 
(big boosts are needed). The CE ratio also provides a benchmark for alternative policies—that is, 
ways to staff these positions that cost less than $11,150 per year. 


Policies to recruit and retain teachers may have diverse consequences. But 
CEA can still be employed as a valid method of evaluation: by focusing on 
one effect, with a ratio expressed in dollars, CEA provides useful evidence 
on the resources needed to deploy teachers optimally. 


Decision-Making With Cost-Effectiveness Analysis 


By itself, the idea of cost effectiveness is straightforward: schools and districts should 
systematically consider how the money they spend helps them achieve their goals. As discussed 
above, however, there are misconceptions as well as barriers that restrict the use of economic 
analysis for decision-making. Here, we revisit these and suggest ways to address them—after the 
analysis part of the CEA has been successfully performed. 
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We adopt the perspective of the analyst considering whether to perform CEA. The analyst 
may be a private consultant, a Chief Business Official, an accountant, or a finance officer—anyone 
who has the skills and experience to perform this type of economic research. The analyst will 
then report the findings to a professional with authority to make decisions about educational 
programs—district-level staff, state officials, and senior school personnel. 


To begin, it is important to consider the misconceptions and barriers so as to ensure they 
will not undermine the value of any economic research before such research Is undertaken. 
We recognize that it may not be possible to overcome these barriers in Some contexts and for 
some decisions; if so, then CEA is not warranted. However, there are a range of possible actions 
an analyst might take to overcome these obstacles. 


District professionals may independently facilitate economic analysis. One step is to 
draw on newly available data. Currently, a “cradle-to-career” data system is being developed in 
California. This system will provide more data for analysis. Districts may use these data directly 
to expedite economic analysis and as benchmarks or diagnostics to identify research priorities. 
A second step is to build capacity so that personnel are prepared to conduct economic analysis. 
Districts and schools already employ many professionals with research and analytical skills such 
that much of the economic analysis can be performed internally; with some practical training 
of their current staff, therefore, some districts may be able to develop capacity to perform CEA 
internally. Other districts may find collaborative partnerships with research agencies to be useful. 


Clarifying Misconceptions 


Economic analysts should take steps to clarify the misconceptions that decision makers 
may have about CEA. Specifically, it should be emphasized that CEA is: 


* motivated to ensure that resources are used in accordance with the districts’ and 
schools’ own objectives; 

* collaborative with education professionals in research design and analysis; 

¢ focused on topics that are integral to education and learning; and 

¢ responsive to institutional, cultural, and political preferences. 


CEA is a method for generating evaluative information. This information helps with—but 
does not enforce—decisions; educational professionals are responsible for making decisions. 
Economic evidence may be subsidiary to other factors, including the need for an equitable 
education system. CEA is a supplement to, not a replacement for, professional experience; 
it involves educators in decisions about efficiency that might typically be left to financial officers. 
In itself, being efficient is not the goal of an education system (or any system); it is a statement 
that the goals of that system are being met optimally. 
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Removing Obstacles to Economic Research 


The most basic barrier to economic research is that analysis takes time and resources to 
perform. Given the scarcity of resources, economic research—particularly if there is no guarantee 
of definitive findings—may not be worthwhile. Ultimately, there is no way to produce research 
without research funding. But there are some responses an analyst might make to concerns 
around this issue. 


One response is to consider what resources might be misallocated if an economic 
evaluation is not performed. For example, California might implement a statewide class 
size reduction policy potentially requiring tens of millions of dollars in funding or resource 
reallocation. If an evaluation subsequently determines that an equivalent educational 
improvement is possible via a policy with a much lower cost (e.g., peer tutoring), then tens of 
millions of dollars have been misallocated. 


A related response is to set threshold criteria for when a CEA is appropriate. The federal 
government mandates economic evaluation when a policy reaches a certain dollar amount. 
A similar mandate might be considered for California's educational policies and interventions. 
For instance, if a district with student enrollment of 5,000 Is deciding to commit more than 
$4 million (e.g., for a new online learning platform), then this value might trigger an economic 
evaluation. (The exact dollar threshold would depend on the district's budget, the capacity to 
perform CEAs, and the cost of performing CEAs.) Any policy below $4 million might be exempt 
from cost-effectiveness evaluation. From a public accountability perspective, it may be considered 
financially prudent to provide an economic evaluation when more than $4 million is being spent. 
Some programs—but not all—would then be subject to economic appraisal. 


A third response is to find out if economic evaluation is actually expensive. It might not 
be, particularly if an impact evaluation has already been undertaken. If a district has evaluated, 
for example, the effects on achievement of an online learning platform or a new textbook, then 
it may be relatively inexpensive to calculate costs and link these to the achievement effects. 
Potentially, CEAs can “piggyback” onto impact or effectiveness evaluations. 


Research itself needs resources; districts may claim they do not have 
sufficient funding. Even as there is a cost to performing economic 
evaluation, there is a cost to not performing it. 


The more fundamental barrier—that research Is not always useful—can be addressed ina 
variety of ways. Most obviously, the analyst needs to perform research that /s relevant to the local 
educational and economic conditions—primarily by engaging with stakeholders to explain the 
value of economic research in helping them address their specific objectives. 
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One way to engage stakeholders is to focus on large-scale policies or priority topics. For 
example, California spends billions of dollars on special education programs and yet almost no 
economic analysis is performed to ensure these programs are cost effective. Other areas where 
CEA may be relevant include: online learning; teacher professional development; early childhood 
education; and counselling services. If the analyst selects projects that are high profile or high 
cost, then stakeholders may be more receptive to evidence on cost effectiveness. 


Economic research should focus on policies and programs that require 
significant funding. The analyst should select programs based on the 
amount of resources used within each program. 


The analyst may rely on existing evidence on the effectiveness of policies and programs 
within the California school system. If existing evidence is being relied on, the analyst should 
formally contextualize that evidence within local educational and economic conditions. For local 
educational conditions, the analyst should consider the baseline characteristics of the districts, 
schools, and students, as well as the mediators and moderators for any relationship between a 
program and outcomes (Weiss et al., 2014). For local economic conditions, the analyst should 
consider the prices of relevant inputs. Because markets exist across localities, these differences 
in local economic conditions may not substantially affect the cost of the intervention. But this 
assumption should be checked. 


Research needs to be externally valid with respect to the educational 
and economic conditions facing the decision maker. The analyst should 
formally investigate the extent to which any results are relevant to the 
conditions faced by decision makers. 


Too often, economic analysis lacks relevance because results are presented in the abstract. 
That is, programs are evaluated as cost effective (or cost ineffective) without any guidance as to 
how a district might be able either to afford or to implement a cost-effective program. If districts 
are committed—either by law or by funding formulae—to provide specific services, then economic 
evaluation may not be helpful (unless it helps influence these commitments). 


The analyst should explicitly consider both how resources might be 
allocated towards a program that has been identified as cost effective and 
who has the authority to decide. 
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Research is not always clearly presented to its audiences. With economic research, the 
analyst should emphasize that the CE ratio is the “critical statistic” that summarizes the findings. 
In this respect, CEA is relatively straightforward compared to other research methods. The analysis 
is solely motivated to calculating a CE ratio; this ratio should convey the overall conclusions from 
the research. It may not be clear to decision makers what the ratio means, so the analyst should 
explicitly explain its implications.’® 


The analyst should focus on reporting the CE ratio and on providing a full 
explanation of what that ratio means. 


The final barrier relates to decision-making authority (or lack thereof). Even as there are 
many stakeholders in education systems, decisions about resource allocation still have to be 
made. Even when funding is strongly historical (depending on past funding), decision makers have 
some discretion to move certain resources towards more efficient programs. If CEA is performed 
before an intervention is selected, there will be more scope for efficient resource allocation. 

CEA may indicate what new resources should be utilized over the long term (e.g., when textbooks 
need restocking or teacher contracts renegotiated). Also, CEA may be useful for accountability 
purposes if the evidence that justifies the status quo is cost effective (or if proposed alternatives are 
not cost effective). Overall, CEA is most clearly related to decisions when it is correctly framed as 

a choice between two or more alternatives. 


The analyst should focus on the comparative feature of CEA; comparisons 
are only meaningful if there are at least two alternatives (even if one is 

the status quo). Thus, CEA allows the analyst to frame the research as a 
genuine choice to be made. 


18 Interpretation of CE ratios is not always straightforward. If there are two CE ratios (e.g., one each for implementing program A and 
program 8), then it is straightforward to infer that the program with the lowest CE ratio is the most cost effective or, more simply, 
that it is preferred (from a cost-effectiveness perspective) over all the available options. But if there is only one CE ratio, then it 

may be difficult to decide. For example, if for $200 per student, California Math generates a 0.08 standard deviation gain in math, 

is that an efficient expenditure? The response to this question is that CEA is comparative: the expenditure is efficient if there is no 
alternative way to generate these gains in math at a lower cost. 
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Conclusions 


CEA is intended to help education professionals make more efficient decisions. But it is also 
rarely practiced—a series of misconceptions and legitimate barriers work against its application. 


Yet, as described here, the CEA method is relatively straightforward and can yield 
conclusions that directly help decision makers. The analytical objective of CEA is deliberately 
simple: to find out which interventions, programs, policies, or reforms are the lowest cost options 
for a given outcome. This simplicity is reflected in the CE ratio, where lower values are regarded 
as more efficient. Simplicity has its weaknesses, especially as educational processes are complex 
and outcomes varied. However, simplicity also has important virtues: CEA is transparent, can be 
applied to most education decisions, and produces results that are clear. Using CEA, education 
professionals may focus on key findings, compare alternatives, and produce evidence that is 
accountable to stakeholders. 


As with all research, economic research needs to produce evidence and results that 
decision makers will use. There are a series of important actions that analysts can take to improve 
the use of economic research. The first of these is to evaluate programs and policies that require 
the most resources. Other actions relate to engagement with education professionals so that 
evidence and results are clear, timely, and relevant to the local context and community. 


CEAs need to be an input into actual decisions made by district leaders and other 
education professionals when allocating resources. To emphasize, district and school leaders 
ultimately make decisions about education policies and programs. By promoting economic 
research—highlighting its importance, clarifying its methodology, and performing valid and useful 
studies—analysts can help make decisions that more closely align with districts’ objectives. 
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